{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "3j57Jh9VIbqo"
   },
   "source": [
    "# Segment Anything Model using `transformers` 🤗 library\n",
    "\n",
    "\n",
    "| <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/sam-beancans.png\" alt=\"Snow\" width=\"300\" height=\"300\"> | <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/sam-dog-masks.png\" alt=\"Forest\" width=\"300\" height=\"300\"> | <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/sam-car-seg.png\" alt=\"Mountains\" width=\"300\" height=\"300\"> |\n",
    "|---------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------|\n",
    "\n",
    "\n",
    "This notebook demonstrates how to use the Segment Anything Model (SAM) to segment objects in images. The model has been released by Meta AI in the paper [Segment Anything Model](https://ai.facebook.com/research/publications/segment-anything/). The original source code can be found [here](https://github.com/facebookresearch/segment-anything)\n",
    "\n",
    "This notebook demonstrates how to use `transformers` to leverage the different usecases of the model. The examples are heavily inspired from the [original notebook of the authors](https://github.com/facebookresearch/segment-anything/blob/main/notebooks/predictor_example.ipynb).\n",
    "\n",
    "As stated by that notebook: \n",
    "> The Segment Anything Model (SAM) predicts object masks given prompts that indicate the desired object. The model first converts the image into an image embedding that allows high quality masks to be efficiently produced from a prompt."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "nel5OwsjCjBh",
    "outputId": "84a261d8-141a-4861-dfba-4596124ebaf2"
   },
   "outputs": [],
   "source": [
    "!pip install -q transformers"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "CsyajN_HDDqp"
   },
   "source": [
    "## Utility functions"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "MkIpaOKVIJKm"
   },
   "source": [
    "Run the cells below to import the needed utility functions for displaying the masks!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "8AkjSITtDBuj"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def show_mask(mask, ax, random_color=False):\n",
    "    if random_color:\n",
    "        color = np.concatenate([np.random.random(3), np.array([0.6])], axis=0)\n",
    "    else:\n",
    "        color = np.array([30/255, 144/255, 255/255, 0.6])\n",
    "    h, w = mask.shape[-2:]\n",
    "    mask_image = mask.reshape(h, w, 1) * color.reshape(1, 1, -1)\n",
    "    ax.imshow(mask_image)\n",
    "\n",
    "\n",
    "def show_box(box, ax):\n",
    "    x0, y0 = box[0], box[1]\n",
    "    w, h = box[2] - box[0], box[3] - box[1]\n",
    "    ax.add_patch(plt.Rectangle((x0, y0), w, h, edgecolor='green', facecolor=(0,0,0,0), lw=2))  \n",
    "\n",
    "def show_boxes_on_image(raw_image, boxes):\n",
    "    plt.figure(figsize=(10,10))\n",
    "    plt.imshow(raw_image)\n",
    "    for box in boxes:\n",
    "      show_box(box, plt.gca())\n",
    "    plt.axis('on')\n",
    "    plt.show()\n",
    "\n",
    "def show_points_on_image(raw_image, input_points, input_labels=None):\n",
    "    plt.figure(figsize=(10,10))\n",
    "    plt.imshow(raw_image)\n",
    "    input_points = np.array(input_points)\n",
    "    if input_labels is None:\n",
    "      labels = np.ones_like(input_points[:, 0])\n",
    "    else:\n",
    "      labels = np.array(input_labels)\n",
    "    show_points(input_points, labels, plt.gca())\n",
    "    plt.axis('on')\n",
    "    plt.show()\n",
    "\n",
    "def show_points_and_boxes_on_image(raw_image, boxes, input_points, input_labels=None):\n",
    "    plt.figure(figsize=(10,10))\n",
    "    plt.imshow(raw_image)\n",
    "    input_points = np.array(input_points)\n",
    "    if input_labels is None:\n",
    "      labels = np.ones_like(input_points[:, 0])\n",
    "    else:\n",
    "      labels = np.array(input_labels)\n",
    "    show_points(input_points, labels, plt.gca())\n",
    "    for box in boxes:\n",
    "      show_box(box, plt.gca())\n",
    "    plt.axis('on')\n",
    "    plt.show()\n",
    "\n",
    "\n",
    "def show_points_and_boxes_on_image(raw_image, boxes, input_points, input_labels=None):\n",
    "    plt.figure(figsize=(10,10))\n",
    "    plt.imshow(raw_image)\n",
    "    input_points = np.array(input_points)\n",
    "    if input_labels is None:\n",
    "      labels = np.ones_like(input_points[:, 0])\n",
    "    else:\n",
    "      labels = np.array(input_labels)\n",
    "    show_points(input_points, labels, plt.gca())\n",
    "    for box in boxes:\n",
    "      show_box(box, plt.gca())\n",
    "    plt.axis('on')\n",
    "    plt.show()\n",
    "\n",
    "\n",
    "def show_points(coords, labels, ax, marker_size=375):\n",
    "    pos_points = coords[labels==1]\n",
    "    neg_points = coords[labels==0]\n",
    "    ax.scatter(pos_points[:, 0], pos_points[:, 1], color='green', marker='*', s=marker_size, edgecolor='white', linewidth=1.25)\n",
    "    ax.scatter(neg_points[:, 0], neg_points[:, 1], color='red', marker='*', s=marker_size, edgecolor='white', linewidth=1.25)\n",
    "\n",
    "\n",
    "def show_masks_on_image(raw_image, masks, scores):\n",
    "    if len(masks.shape) == 4:\n",
    "      masks = masks.squeeze()\n",
    "    if scores.shape[0] == 1:\n",
    "      scores = scores.squeeze()\n",
    "\n",
    "    nb_predictions = scores.shape[-1]\n",
    "    fig, axes = plt.subplots(1, nb_predictions, figsize=(15, 15))\n",
    "\n",
    "    for i, (mask, score) in enumerate(zip(masks, scores)):\n",
    "      mask = mask.cpu().detach()\n",
    "      axes[i].imshow(np.array(raw_image))\n",
    "      show_mask(mask, axes[i])\n",
    "      axes[i].title.set_text(f\"Mask {i+1}, Score: {score.item():.3f}\")\n",
    "      axes[i].axis(\"off\")\n",
    "    plt.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "ksFGk69rDF2q"
   },
   "source": [
    "## Model loading"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "G61pAbslIPUS"
   },
   "source": [
    "Use the `from_pretrained` method on the `SamForMaskGeneration` class to load the model from the Hub! For the sake of this demonstration we will use the `vit-huge` checkpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 113,
     "referenced_widgets": [
      "9b6e632cd863437e8d854d46c6d01317",
      "10c99915753f45cf9604001fc52354b1",
      "fa09cbfb899240bf9e9193b59cbc15ba",
      "598e03bc3124400cb904a9889d15d13e",
      "d6c65a83e6c743d295ec1a26040b28d5",
      "e06a176fa3f74e788d5ef71d5236e98b",
      "82b3eb150fcb4824a5d096c91521848a",
      "cc86bfe38d314dcaa689a21f34aa4fcd",
      "fce27908b6df42e39eafc03ef3864e4d",
      "b8aea01e3c7b41e1b00fca07d893427b",
      "f17344a83b2d46efbb28b88e6f12ce7b",
      "3563127c75de4a389926a6fa0eabc969",
      "eace53a8766f4809952914f531e5ee02",
      "8e6e2f30673e4830a26d1ba79aab0ecc",
      "2a379119b0b44e1dbdcfd6644a010838",
      "601b421ed1964d57bd251c2990e7134e",
      "79e22be273c741269783df9168a2b4f3",
      "a12fa82ddccc48f7810bc01ed9db5cc8",
      "02382a9cdbe74e7396866628a697f980",
      "daa91e593d384fd1ae29237d189bbf4e",
      "2dc9405a3a3e455dbea0728674ca9fed",
      "22cd185554ac49d391ddbbc65b999d3c",
      "c3220c2007234215a4521794fe8b1b7b",
      "99c8363974d34f338cbe489c6591a998",
      "338fe470212c41228527a67c09ad2103",
      "b326c650b7ee47c79d9b200c2f034f30",
      "379ade663ec749f496206b59f1f40925",
      "70761604c98c4d2fb827f5c82d072092",
      "cf154b44adac4291971aab832dfee860",
      "9d501254de13486f8e8bec587e52660a",
      "93457f7957f7434ea820b756f09d7346",
      "d61125039fa24a24bbd5f418e0ce4c44",
      "6002dafc2cb841d198010204400d4418"
     ]
    },
    "id": "P44NSy-ADGph",
    "outputId": "d6e72f37-4412-43a8-e49e-a6b4e3e7ad89"
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "from transformers import SamModel, SamProcessor\n",
    "\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "model = SamModel.from_pretrained(\"facebook/sam-vit-huge\").to(device)\n",
    "processor = SamProcessor.from_pretrained(\"facebook/sam-vit-huge\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "UlcChrbpDORW"
   },
   "source": [
    "## Run predictions"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "9iKYKiNK9WHU"
   },
   "source": [
    "Let's deeply dive into how you can run different type of predictions, given different inputs. You will see how to\n",
    "\n",
    "- Generate segmentation masks given a 2D localization\n",
    "- Generate segmentation masks per given localization (one prediction per 2D point)\n",
    "- Generate segmentation masks given a bounding box\n",
    "- Generate segmentation masks given a bounding box and a 2D points\n",
    "- Generate multiple segmentatation masks per image"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "rvCNeRU2DJZZ"
   },
   "source": [
    "### Load the example image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 413
    },
    "id": "eHd4QBdtDKtf",
    "outputId": "42afcc54-deac-42d3-80d2-8404188215ff"
   },
   "outputs": [],
   "source": [
    "from PIL import Image\n",
    "import requests\n",
    "\n",
    "img_url = \"https://huggingface.co/ybelkada/segment-anything/resolve/main/assets/car.png\"\n",
    "raw_image = Image.open(requests.get(img_url, stream=True).raw).convert(\"RGB\")\n",
    "\n",
    "plt.imshow(raw_image)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "zmybDSsSDNQD"
   },
   "source": [
    "## Step 1: Retrieve the image embeddings\n",
    "\n",
    "In order to avoid computing multiple times the same image embeddings, we will compute it only once, and use these embeddings to directly feed them to the model for faster inference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "PBBZstdyDULA"
   },
   "outputs": [],
   "source": [
    "inputs = processor(raw_image, return_tensors=\"pt\").to(device)\n",
    "image_embeddings = model.get_image_embeddings(inputs[\"pixel_values\"])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "UQ8meq5mDYQ1"
   },
   "source": [
    "## Usecase 1: Feed a set of 2D points to predict a mask\n",
    "\n",
    "Let's first focus on the first classic usecase of SAM. You can feed the model a set of 2D points to predict a segmentation mask. The more you provide 2D points, the better the resulting mask will be. \n",
    "\n",
    "In this example, let's try to predict the mask that corresponds to the top left window of the parked car.\n",
    "\n",
    "The input points needs to be in the format:\n",
    "\n",
    "`nb_images, nb_predictions, nb_points_per_mask, 2`\n",
    "\n",
    "With SAM you can either predict a single prediction given multiple points, or a prediction per point. This is denoted by `nb_predictions` dimension. We will see in the next sections how to perform this type of prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 582
    },
    "id": "N5x_Ni12DbLP",
    "outputId": "f8e01ecb-d0ec-4276-d9a2-3b02d7969a20"
   },
   "outputs": [],
   "source": [
    "input_points = [[[450, 600]]]\n",
    "show_points_on_image(raw_image, input_points[0])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "jhdTFCmLKywE"
   },
   "source": [
    "For that, simply pass the raw image, the points\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "3fIB9JtOKnuw"
   },
   "outputs": [],
   "source": [
    "inputs = processor(raw_image, input_points=input_points, return_tensors=\"pt\").to(device)\n",
    "# pop the pixel_values as they are not neded\n",
    "inputs.pop(\"pixel_values\", None)\n",
    "inputs.update({\"image_embeddings\": image_embeddings})\n",
    "\n",
    "with torch.no_grad():\n",
    "    outputs = model(**inputs)\n",
    "\n",
    "masks = processor.image_processor.post_process_masks(outputs.pred_masks.cpu(), inputs[\"original_sizes\"].cpu(), inputs[\"reshaped_input_sizes\"].cpu())\n",
    "scores = outputs.iou_scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 287
    },
    "id": "9Nvf5Zg6QafA",
    "outputId": "cfe69d81-7608-405c-c638-86f68c52a909"
   },
   "outputs": [],
   "source": [
    "show_masks_on_image(raw_image, masks[0], scores)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "Y--Hcci-RtPl"
   },
   "source": [
    "As you can see, the predicted masks are sorted in their IoU score order. The first mask indeed seems to correspond to the mask of the top right window of the parked car."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "EDTZPfZIRngy"
   },
   "source": [
    "You can also feed a set of points to predict a single mask. Let's try to predict a mask, given two points"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 582
    },
    "id": "JIXnoE0xRmHK",
    "outputId": "2926232c-bd8b-4d52-c425-ac21e0a9c884"
   },
   "outputs": [],
   "source": [
    "input_points = [[[550, 600], [2100, 1000]]]\n",
    "show_points_on_image(raw_image, input_points)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "C7nG5LLDS5q1"
   },
   "outputs": [],
   "source": [
    "inputs = processor(raw_image, input_points=input_points, return_tensors=\"pt\").to(device)\n",
    "# pop the pixel_values as they are not neded\n",
    "inputs.pop(\"pixel_values\", None)\n",
    "inputs.update({\"image_embeddings\": image_embeddings})\n",
    "\n",
    "with torch.no_grad():\n",
    "    outputs = model(**inputs)\n",
    "\n",
    "masks = processor.image_processor.post_process_masks(outputs.pred_masks.cpu(), inputs[\"original_sizes\"].cpu(), inputs[\"reshaped_input_sizes\"].cpu())\n",
    "scores = outputs.iou_scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 287
    },
    "id": "6Grfb__rS945",
    "outputId": "9445b4f7-f36b-4ad5-96d1-22b70efa05ae"
   },
   "outputs": [],
   "source": [
    "show_masks_on_image(raw_image, masks[0], scores)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "FWMuHSAFhMXY"
   },
   "source": [
    "## Usecase 2: Predict segmentations masks using bounding boxes"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "QWnYWTZFhQvh"
   },
   "source": [
    "It is possible to feed bounding boxes to the model to predict segmentation masks of the object of interest in that region.\n",
    "\n",
    "The bounding box needs to be a list of points, corresponding to the flattened coordinates of the top left point, and bottom right point of the bounding box. Let's look at an example below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 582
    },
    "id": "pv6eGSOPhqsI",
    "outputId": "b3ad99a3-290a-4768-9b1b-541d97b2236c"
   },
   "outputs": [],
   "source": [
    "input_boxes = [[[650, 900, 1000, 1250]]]\n",
    "\n",
    "show_boxes_on_image(raw_image, input_boxes[0]) "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "uKTJVdr3ht0l"
   },
   "source": [
    "We will try to segment the wheel that is present inside the bounding box! For that just run the following snippet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "VkuYT4qSh4OQ",
    "outputId": "d5db92b4-f7c8-44d4-bb1c-8f70de670575"
   },
   "outputs": [],
   "source": [
    "inputs = processor(raw_image, input_boxes=[input_boxes], return_tensors=\"pt\").to(device)\n",
    "inputs[\"input_boxes\"].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "v_RjB6xjh6Za"
   },
   "outputs": [],
   "source": [
    "inputs.pop(\"pixel_values\", None)\n",
    "inputs.update({\"image_embeddings\": image_embeddings})\n",
    "\n",
    "with torch.no_grad():\n",
    "    outputs = model(**inputs)\n",
    "\n",
    "masks = processor.image_processor.post_process_masks(outputs.pred_masks.cpu(), inputs[\"original_sizes\"].cpu(), inputs[\"reshaped_input_sizes\"].cpu())\n",
    "scores = outputs.iou_scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 287
    },
    "id": "tKh-HY1AiBiv",
    "outputId": "8f08b3cf-7915-4e2c-ea90-7eb9d58db82b"
   },
   "outputs": [],
   "source": [
    "show_masks_on_image(raw_image, masks[0], scores)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "NOfzmiLwiHk9"
   },
   "source": [
    "It is possible to feed multiple boxes, however, this will lead to having one prediction per bounding box. i.e., you cannot combine multiple bounding boxes to get a single prediction. However, you can combine points and bounding boxes to get a prediction, and we will cover that in the next section"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "uMPc-vBbiZWo"
   },
   "source": [
    "## Usecase 3: Predict segmentation masks given points and bounding boxes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 582
    },
    "id": "B5_nG1pjicvX",
    "outputId": "96b88443-0247-4059-9c29-805737e0356e"
   },
   "outputs": [],
   "source": [
    "input_boxes = [[[650, 900, 1000, 1250]]]\n",
    "input_points = [[[820, 1080]]]\n",
    "\n",
    "show_points_and_boxes_on_image(raw_image, input_boxes[0], input_points[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "pXfn6e9qjz28"
   },
   "outputs": [],
   "source": [
    "inputs = processor(raw_image, input_boxes=[input_boxes], input_points=[input_points], return_tensors=\"pt\").to(device)\n",
    "\n",
    "inputs.pop(\"pixel_values\", None)\n",
    "inputs.update({\"image_embeddings\": image_embeddings})\n",
    "\n",
    "with torch.no_grad():\n",
    "    outputs = model(**inputs)\n",
    "\n",
    "masks = processor.image_processor.post_process_masks(outputs.pred_masks.cpu(), inputs[\"original_sizes\"].cpu(), inputs[\"reshaped_input_sizes\"].cpu())\n",
    "scores = outputs.iou_scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 287
    },
    "id": "VJoXPp7qo6Mi",
    "outputId": "cedc0ae8-65fa-402a-80ba-e531cbb8c1c9"
   },
   "outputs": [],
   "source": [
    "show_masks_on_image(raw_image, masks[0][0], scores[:, 0, :])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "Du9ETa6DpHr1"
   },
   "source": [
    "You can also pass points with a label to segment out that region. Let us have a deeper look below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 582
    },
    "id": "NEwSEeJXjw2y",
    "outputId": "989be05e-7b2b-4ef3-d9f7-da566e50762b"
   },
   "outputs": [],
   "source": [
    "input_boxes = [[[650, 900, 1000, 1250]]]\n",
    "input_points = [[[820, 1080]]]\n",
    "labels = [0]\n",
    "\n",
    "show_points_and_boxes_on_image(raw_image, input_boxes[0], input_points[0], labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Ry5lXZN-p0fg"
   },
   "outputs": [],
   "source": [
    "input_boxes = [[[620, 900, 1000, 1255]]]\n",
    "input_points = [[[820, 1080]]]\n",
    "labels = [[0]]\n",
    "inputs = processor(raw_image, input_boxes=[input_boxes], input_points=[input_points], input_labels=[labels], return_tensors=\"pt\").to(device)\n",
    "\n",
    "inputs.pop(\"pixel_values\", None)\n",
    "inputs.update({\"image_embeddings\": image_embeddings})\n",
    "\n",
    "with torch.no_grad():\n",
    "    outputs = model(**inputs)\n",
    "\n",
    "masks = processor.image_processor.post_process_masks(outputs.pred_masks.cpu(), inputs[\"original_sizes\"].cpu(), inputs[\"reshaped_input_sizes\"].cpu())\n",
    "scores = outputs.iou_scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 287
    },
    "id": "cenkySXQp7DD",
    "outputId": "9cc1b860-a5fc-4f80-e871-0833c894f7b7"
   },
   "outputs": [],
   "source": [
    "show_masks_on_image(raw_image, masks[0][0], scores[:, 0, :])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "oWcGnzHtzICx"
   },
   "source": [
    "As you can see, the model managed to \"ignore\" the component that was specified by the point with the label `0`."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "IeTLn8O4TYXF"
   },
   "source": [
    "## Usecase 4: Predict multiple masks per image\n",
    "\n",
    "With SAM, you can also predict multiple masks per image. You can achieve that in two possible scenarios\n",
    "\n",
    "- Feed multiple points in the `nb_predictions` dimension\n",
    "- Feed multiple bounding boxes to the same image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 582
    },
    "id": "SmFVP2sIUDAR",
    "outputId": "c3ba912d-5433-40d9-9c73-485ad0261029"
   },
   "outputs": [],
   "source": [
    "input_points = [[[850, 1100], [2250, 1000]]]\n",
    "show_points_on_image(raw_image, input_points)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "Pdx778ExUovK"
   },
   "source": [
    "### Sub-usecase 1: one prediction per point"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "ng8LtcHTYsxS"
   },
   "source": [
    "To benefit from what we have described in the first bullet point, just change the input array to"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "QzmnmFP7UZts"
   },
   "outputs": [],
   "source": [
    "input_points = [[[850, 1100]], [[2250, 1000]]]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "h0yc7-uLUyid"
   },
   "source": [
    "In order to add the desired dimension, and pass it to the `SamProcessor`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "o3INMCRNUw0m",
    "outputId": "f170d59c-172d-45cd-8d3c-1df456c0ecd4"
   },
   "outputs": [],
   "source": [
    "input_points = [[[[850, 1100]], [[2250, 1000]]]]\n",
    "inputs = processor(raw_image, input_points=input_points, return_tensors=\"pt\").to(device)\n",
    "inputs[\"input_points\"].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "gNmlW6SoVUIN"
   },
   "outputs": [],
   "source": [
    "inputs.pop(\"pixel_values\", None)\n",
    "inputs.update({\"image_embeddings\": image_embeddings})\n",
    "\n",
    "with torch.no_grad():\n",
    "    outputs = model(**inputs)\n",
    "\n",
    "masks = processor.image_processor.post_process_masks(outputs.pred_masks.cpu(), inputs[\"original_sizes\"].cpu(), inputs[\"reshaped_input_sizes\"].cpu())\n",
    "scores = outputs.iou_scores"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "H35bShwLW2ts"
   },
   "source": [
    "Let's print the shapes of the output to understand better what is going on"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "KcNPu4meW2Nc",
    "outputId": "ebc49079-3c76-4a43-ae49-7461b267e90b"
   },
   "outputs": [],
   "source": [
    "scores.shape"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "131QhQdAW6eM"
   },
   "source": [
    "Here the first dimension corresponds to the image batch size, the second dimension corresponds to the `nb_predictions` dimension. And the last dimension is the number of predicted masks **per prediction** , and it is set to 3 by default according to the official implementation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 287
    },
    "id": "UmYgPcK4Vdow",
    "outputId": "7541fbd8-8e4f-4319-9349-fcaf69ccfc37"
   },
   "outputs": [],
   "source": [
    "show_masks_on_image(raw_image, masks[0][0], scores[:, 0, :])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 287
    },
    "id": "wuIvSJBYV08J",
    "outputId": "e60f5a08-94ee-4681-fca0-f2160941c451"
   },
   "outputs": [],
   "source": [
    "show_masks_on_image(raw_image, masks[0][1], scores[:, 0, :])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "Uiw2wUDRYgKs"
   },
   "source": [
    "### Sub-usecase 2: Feed multiple bounding boxes to the same image"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "zUytan90Y26q"
   },
   "source": [
    "You can also feed multiple bounding boxes to the same image and get one prediction per bounding box. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 582
    },
    "id": "G9LmqfxvbpMw",
    "outputId": "ebaee8ec-f07e-4b84-b578-1cc2e94c68ae"
   },
   "outputs": [],
   "source": [
    "input_boxes = [[[650, 900, 1000, 1250], [2050, 800, 2400, 1150]]]\n",
    "\n",
    "show_boxes_on_image(raw_image, input_boxes[0]) "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "5t3PtwgSdysQ"
   },
   "source": [
    "Just pass the input boxes as follows, to match the convention of the processor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "CmrVrYb5dodB",
    "outputId": "fa377dcf-0f7b-419d-aeeb-6e0fa1a8c1be"
   },
   "outputs": [],
   "source": [
    "input_boxes = [[[650, 900, 1000, 1250], [2050, 800, 2400, 1150]]]\n",
    "inputs = processor(raw_image, input_boxes=input_boxes, return_tensors=\"pt\").to(device)\n",
    "inputs[\"input_boxes\"].shape"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "A189dbgPd6sG"
   },
   "source": [
    "This time, let's just output a single mask per box, for that we can just pass `multimask_output=False` in the forward pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "XqFmNWhrd3wu"
   },
   "outputs": [],
   "source": [
    "inputs.pop(\"pixel_values\", None)\n",
    "inputs.update({\"image_embeddings\": image_embeddings})\n",
    "\n",
    "with torch.no_grad():\n",
    "    outputs = model(**inputs, multimask_output=False)\n",
    "\n",
    "masks = processor.image_processor.post_process_masks(outputs.pred_masks.cpu(), inputs[\"original_sizes\"].cpu(), inputs[\"reshaped_input_sizes\"].cpu())\n",
    "scores = outputs.iou_scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "FVrDRktleZDI",
    "outputId": "affe0b9d-73d3-4cd8-a60e-0d4d8879d5b8"
   },
   "outputs": [],
   "source": [
    "scores.shape"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "PnXu5yU_e3s7"
   },
   "source": [
    "As you can see, here we have predicted 2 masks in total! Let's check them now"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 411
    },
    "id": "ZXlwHn6reDru",
    "outputId": "905c5cb1-bc72-474f-ec34-669cec91050a"
   },
   "outputs": [],
   "source": [
    "show_masks_on_image(raw_image, masks[0], scores)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [
    "CsyajN_HDDqp"
   ],
   "provenance": []
  },
  "gpuClass": "standard",
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "02382a9cdbe74e7396866628a697f980": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "10c99915753f45cf9604001fc52354b1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e06a176fa3f74e788d5ef71d5236e98b",
      "placeholder": "​",
      "style": "IPY_MODEL_82b3eb150fcb4824a5d096c91521848a",
      "value": "Downloading (…)lve/main/config.json: 100%"
     }
    },
    "22cd185554ac49d391ddbbc65b999d3c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "2a379119b0b44e1dbdcfd6644a010838": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2dc9405a3a3e455dbea0728674ca9fed",
      "placeholder": "​",
      "style": "IPY_MODEL_22cd185554ac49d391ddbbc65b999d3c",
      "value": " 2.56G/2.56G [01:05&lt;00:00, 39.3MB/s]"
     }
    },
    "2dc9405a3a3e455dbea0728674ca9fed": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "338fe470212c41228527a67c09ad2103": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9d501254de13486f8e8bec587e52660a",
      "max": 466,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_93457f7957f7434ea820b756f09d7346",
      "value": 466
     }
    },
    "3563127c75de4a389926a6fa0eabc969": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_eace53a8766f4809952914f531e5ee02",
       "IPY_MODEL_8e6e2f30673e4830a26d1ba79aab0ecc",
       "IPY_MODEL_2a379119b0b44e1dbdcfd6644a010838"
      ],
      "layout": "IPY_MODEL_601b421ed1964d57bd251c2990e7134e"
     }
    },
    "379ade663ec749f496206b59f1f40925": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "598e03bc3124400cb904a9889d15d13e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b8aea01e3c7b41e1b00fca07d893427b",
      "placeholder": "​",
      "style": "IPY_MODEL_f17344a83b2d46efbb28b88e6f12ce7b",
      "value": " 6.53k/6.53k [00:00&lt;00:00, 208kB/s]"
     }
    },
    "6002dafc2cb841d198010204400d4418": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "601b421ed1964d57bd251c2990e7134e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "70761604c98c4d2fb827f5c82d072092": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "79e22be273c741269783df9168a2b4f3": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "82b3eb150fcb4824a5d096c91521848a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "8e6e2f30673e4830a26d1ba79aab0ecc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_02382a9cdbe74e7396866628a697f980",
      "max": 2564565013,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_daa91e593d384fd1ae29237d189bbf4e",
      "value": 2564565013
     }
    },
    "93457f7957f7434ea820b756f09d7346": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "99c8363974d34f338cbe489c6591a998": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_70761604c98c4d2fb827f5c82d072092",
      "placeholder": "​",
      "style": "IPY_MODEL_cf154b44adac4291971aab832dfee860",
      "value": "Downloading (…)rocessor_config.json: 100%"
     }
    },
    "9b6e632cd863437e8d854d46c6d01317": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_10c99915753f45cf9604001fc52354b1",
       "IPY_MODEL_fa09cbfb899240bf9e9193b59cbc15ba",
       "IPY_MODEL_598e03bc3124400cb904a9889d15d13e"
      ],
      "layout": "IPY_MODEL_d6c65a83e6c743d295ec1a26040b28d5"
     }
    },
    "9d501254de13486f8e8bec587e52660a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a12fa82ddccc48f7810bc01ed9db5cc8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b326c650b7ee47c79d9b200c2f034f30": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d61125039fa24a24bbd5f418e0ce4c44",
      "placeholder": "​",
      "style": "IPY_MODEL_6002dafc2cb841d198010204400d4418",
      "value": " 466/466 [00:00&lt;00:00, 12.6kB/s]"
     }
    },
    "b8aea01e3c7b41e1b00fca07d893427b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c3220c2007234215a4521794fe8b1b7b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_99c8363974d34f338cbe489c6591a998",
       "IPY_MODEL_338fe470212c41228527a67c09ad2103",
       "IPY_MODEL_b326c650b7ee47c79d9b200c2f034f30"
      ],
      "layout": "IPY_MODEL_379ade663ec749f496206b59f1f40925"
     }
    },
    "cc86bfe38d314dcaa689a21f34aa4fcd": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cf154b44adac4291971aab832dfee860": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d61125039fa24a24bbd5f418e0ce4c44": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d6c65a83e6c743d295ec1a26040b28d5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "daa91e593d384fd1ae29237d189bbf4e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "e06a176fa3f74e788d5ef71d5236e98b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "eace53a8766f4809952914f531e5ee02": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_79e22be273c741269783df9168a2b4f3",
      "placeholder": "​",
      "style": "IPY_MODEL_a12fa82ddccc48f7810bc01ed9db5cc8",
      "value": "Downloading pytorch_model.bin: 100%"
     }
    },
    "f17344a83b2d46efbb28b88e6f12ce7b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "fa09cbfb899240bf9e9193b59cbc15ba": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cc86bfe38d314dcaa689a21f34aa4fcd",
      "max": 6532,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_fce27908b6df42e39eafc03ef3864e4d",
      "value": 6532
     }
    },
    "fce27908b6df42e39eafc03ef3864e4d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
